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The nonlinear conjugate gradient algorithm is one of the effective algorithms 
for optimization since it has low storage and simple structure properties. The 
coefficient conjugate is the basis of conjugate gradient algorithms with the 
desirable conjugate property. In this manuscript, we have derived a new 
second order information for the Hessian from objective function, which can 


give anew search direction. Based on new search direction, we have proposed 
the update formula interesting and nonlinear conjugate gradient method. 
Keywords: Under wolfe line search and mild assumptions on objective function, the 
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1. INTRODUCTION 

Images are often corrupted by impulse noise. Salt-and-pepper and random-valued impulse noise are 
removed using two-phase techniques. The first phase employs decision-based median filters to identify pixels 
that are prone to noise corruption (noise candidates). These noise candidates are restored in the second phase 
using a detail-preserving regularization approach that preserves edges and noise-free pixels. Let X be the 
genuine image and A = {1,2,3,..... M} x {1,2,3,..... N} be the index set of X. The collection of indices of the 
noise pixels observed in the first phase is denoted by N C A. The noise pixels are then recovered in the second 
phase by (1): 


F,(u) = Yia,pen [le - yij| +22 x Sit s?/)| (1) 


where B is the regularization parameter,S};; we Lomnyer; awe Og — Venn de St; = Lomnyer; jaw dali; — 
Ymn) and edge-preserving potential function is dg = Va+x?,a>0. Let P,; be the set of four closest 
neighbors of the pixel at location (i,j) € A, y;,; the image's observed pixel value at position (i,j), , and uj; = 
[u;, ase a lexicographically ordered column vector of length c. The number of elements in N is represented 
by the letter c. In actuality, the smooth data-fitting term is eliminated from the minimization, and only noisy 
pixels are restored. The smooth functional that results is as shown in (2). 


Fy(u) = Da jyen|(2 x Si + 82, (2) 
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For more details see Xue ef al. [1] and Yu et al. [2]. Our aim is to minimize a function of 1 variables by using 
iteration methods: 


Minf(u) , ue R% (3) 


where f is smooth and its gradient g,4, 1s available. Many practical problems can be translated into the 
problems, see Dai and Wen [3] and Hajmohammadi [4]. 

One of the most widely remarkable and powerful used minimization functions is conjugate gradient 
methods. Begins the minimization process with an initial estimate x, and an initial search direction dy) = —Qo. 
As we noted that the iterative formula given by (4): 


Uns1 = Ux + andy (4) 


where a@;, is a step length generated by a suitable line search and d;, is the search direction, see [5]. Obviously, 
the approximate adequate step size associated to quadratic function f;, is (5). 


T 
— ~9Ktk 
Ok = ody ©) 
For general nonlinear functions, it is necessary to use an iterative procedure [6]. We use the typical Wolfe 
criteria to determine the step length throughout our operation, as shown in (6) and (7): 


f (Ug + adi) S f (Up) + Say. gf dy (6) 
dig (Uy + dy) = 0 dh gy (7) 


where 0 < 6 <o <1, Moghrabi [7]. In conjugate gradient methods, the search directions can be defined 
recursively: 


deat = —Gxs1 + Besx (8) 


where the coefficient 6; is so chosen that d, and d,,, must fulfill the conjugacy property. In practical 
computations promising, the CG methods include suggested by Fletcher and Reeves (FR) [8] and Dai and Yuan 
(DY) [9] is believed of the efficient methods, respectively given by: 


T 
FR — Gk4+19k4+1 


DY _ Ths I k+1 9 
k T = (9) 
IKIK 


at aby 


or via other formulaes (e.g. see [4], [10]-[12]). Some of conjugate gradient (CG) approaches are numerically 
successful, while the others are conceptually successful. 

Later, if the point x; is close enough to a local minimizer u* in pure Newton's technique, the search 
direction of good to follow, that is, dy44 = —Qe419x-41, 18 determined. Nazareth [13], inspired by this, rewrites 
in (5). 


—QitiGk+1 = —Ins1 + Bese (10) 


On this guideline, this work is improved to investigate ideas from Newton updates, in order to build definite 
new method for conjugate gradient methods. They proved to be really effective in practice and showed mature 
convergence properties. This idea was presented as a technical method in some manuscripts, for example, to 
maximize in the advantages of the original respective conjugate gradient methods [7], [14]-[16]. To introduce 
a new method, we will find the Hessian approximation of the minimum of a function f(u), which give a new 
search direction and choose the coefficient conjugate satisfies above relation. 


2. OUR NEW COEFFICIENT CONJUGATE 
A very important coefficient conjugate in minimization is that for conjugate gradient method. In order 
to deriving a new coefficient conjugate, we exposed a second order Taylor series as: 


F(X) = f eas) + haa & — Kear) + 5% — Hee1)"O Xess) (% — X41) (1) 
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can see derivative as (12). 

Gti = Get WXe+1)Sk (12) 
A good second order curvature information, we obtained (13). 

SEQ (Xe41) Se = Cera — fc) — 3/25h Ge (13) 
The yielded matrix Q(%,41) can be as (14). 


_ Fhet1-Si)=3/ 25K Ik 


Q(Xr41) ce (14) 
KSKk 
Putting Q(x,41) in (10) we get (15). 
ee Te 
2 1 ae) ae ; 
Br ( CFeti1—Se-3/25h 9k) SEK >) 
In addition by (14) is gained (16). 
7. TE 
2 1 ae) a i 
Bx ( CFeti-Si+3/254 Vk) SkVK (16) 


To ensure the sufficient descent condition, we will do some algebra manipulations on above formula, we get: 


T 
1 llyxll? ) 
=—— -wW =—S 17 
Bx r(% Tyg hy aed (17) 
where: 
= (sk) Ez SRSk | (18a) 
IIvell? Lsfsie Feta-Fe)-3/25h 9x 
_ (SkYK) Re : Sk | (18b) 
IIvill? Lsfsie Peta—Fa)+3/25, Vx 


this formulas will assist us to proving that the direction is satisfied the sufficient descent property. We denote 
by binarized neural networks (BNN) and block truncated-Newton (BTN) the methods defined by (17). We 
introduced the BNN and BTN methods by the following algorithm, which would be used for its convergence 
analysis. New algorithm (BNN and BTN Algorithms): 

- Stage 1. Give initial point u,. Set k = 1 and d, = —g,. If ||g,|| < 10~°, then stop. 

- Stage 2. Evaluate a, satisfying the Wolfe conditions (6) and (7). 

- Stage 3. Let Ugg, = Ug + yd. If |l9x41|| < 107°, then stop. 

- Stage 4. Evaluate f;, by the formulae (17) then generate d,4, by (8). 

- Stage 5. Set k = k + 1 and continue with step 2. 

Theorem 1. 


aS 5 
Let Se Vier Grrr © R" Be ER and BRYN = r-(ve = oy al sx) Durr TE se¥e #0, then 
KV k 
1 
dha Ges1 S—[1—-] Mousa? 
Proof : 
Since dy = —gJo, we have g3dy = —||_goll?, which satisfy (17). Multiplying (8) by gz41, we have: 


Tha lly xl? 
sLyx (sh yR)? 


Aha1Gu+1 = —ll9n+r ll? + ( ghs15e) SEIk+1 (19) 


Yielding: 
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(Thess Vie SKI +1) SKY KI=M Ie rll? (Sh VK)? — ly ell? her Sk)” 
rs rr core ec (20) 
now, by applying: 
1 
WS ae (SEV) Geer and v = V2D(Gh 415K) Ve (21) 
into the inequality: 
1 
w?v <= (wll? + lvl?) (22) 
we can obtain: 
apa 
(Ges YeN(SE Guerre) S=|X CSP i)? MGieaall? + 20 (SF Gea1)? llvell?| (23) 


therefore by (23) and (20), we have that: 


1 
< law thy) llansall? Howl (sk acta) lye? 


div Ge+1 S ae (24) 
therefore, we get: 
1 
de s9eer S—[1- FS] lle? (25) 


which completes our proof. In BTN method similar to the proof done in BNN. 


3. CONVERGENCE ANALYSIS 

To understand the global convergence theorem of the BNN technique, we must first understand the 
assumptions listed below. 1. The. 2 = {x € R"/f (x) S f(x,)} is a bounded level set. 2. In some neighborhood 
A of 2, the gradient of function g is Lipschitz continuous, namely, there exists a constant L > 0 such that: 


llg(o)-g@Il SL llo-tlLvi0EA (26) 
We demonstrate the Dai et al. [17] theorem it is very important for deducing global convergence. 


Lemma 1. 
Let x, be generated by (4), d, satisfy descent property and a; be satisfy (6)-(7). If: 


1 
Dk20 7a, 2 male = (27) 
then: 


limysoo inf|| 9x41 (28) 
We introduced our main theorem of this paper. 
Therorem 2. 


Consider the method formed by (8), (17) and f(x) satisfies Assumption 1 and 2. If step size a, 
satisfies wolfe conditions (6) and (7), then we obtain: 


Lim inf llgull = 0 (29) 


Proof: 
We may deduce the following from (8) and the definition of 6, given by (17) 


Wdiesall = W-gieaa + Be dell S Igual + 
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2 
Lyla qlee Zeeall> lbs 


Isxllllye d 
ale | . : 
Uyeclllox¢al+Olloxss lilly cll 
< SE —=vi |la 
deel esata | 
at wow ant+1+o 
< [1424 2] Iigessll s [= ?] Ngai (30) 
kk hE 
This relationship demonstrates that: 
1 ak 1 
—— >. _ — 
Duet ae 2 (ees) sda P= (31) 
we obtain lim inf \lgx|| = 0 from Lemma 1, which is identical to lim|l 9x\|| = Ofor a uniformly convex 
00 00 


function. In BTN way, one can find its proof similar to the proof done in BNN. 


4. NUMERICAL RESULT 

The goal of the results of applying the proposed BNN and FR methods to test images (Lena, House, 
Cameraman and Elaine), will be reported in the Table 1. is to analyze the practical performance of Algorithms. 
The method was implemented in MATLAB on a PC-type computer with the stoping criterion: 


Per RY < 10-* and [If (up)I] < 10-41 + [fF ue) (32) 


more details can be found in [18]-[24]. The examined pictures are presented in Table 1, and the comprehensive 
numerical results of our testing are reported in Table 1, where NI, NF, peak signal to noise ratio (PSNR) denotes 
the number of iterations, function evaluations, and PSNR, which is defined as: 


2552 


PSNR = 10 log,9 ——————> 33 
J10 jul =i (33) 
; vee 7 ; 
where pixel values for the restored and original images are denoted by uj j and u;, ‘p respectively. 
Table1. Performance of FR, BNN and BTN algorithms 
Image Noise level r (%) FR-Method BNN-Method BTN-Method 
NI NF PSNR(dB) NI NF PSNR(dB) NI NE _ PSNR (dB) 
Le 50 82 153 30.5529 2 8 37.7683 2 8 37.8792 
70 81 155 27.4824 2 8 36.6362 2 8 36.5749 
90 108 211 22.8583 9 29 29.5153 2 8 35.6453 
ho 50 52 53 30.6845 10 34 38.0645 3 14 40.588 
70 63 116 31.2564 12 33 34.9569 4 14 40.2693 
90 111 214 25.287 12 33 31.6781 4 14 38.94 
El 50 35 36 33.9129 10 35 34.6177 3 17 36.4966 
70 38 39 31.864 12 41 33.32 3 17 35.667 
90 65 114 28.2019 14 47 = 32.1012 4 21 33.935 
e512 50 59 87 35.5359 10 31 39.1136 3 12 42.176 
70 78 142 30.6259 12. 39 35.7245 3 14 39.7155 
90 121 236 24.3962 12. 42 33.333 + 19 37.3434 


Figures 1-4 show the obtained results by denoised images. Figure 1(a), Figure 2(a), Figure 3(a), and Figure 
4(a) are the images corrupted with 70% salt-and-pepper noise; Figure 1(b), Figure 2(b), Figure 3(b) and Figure 
4(b) are results of FR method; Figure 1(c), Figure 2(c) and Figure 3(c), and Figure 4(c) are results of the BNN 
method; Figure 1(d), Figure 2(d), Figure 3(d) and Figure 4(d) are results of the BTN method. Take everything 
together, the numerical experiments show that the proposed method performs well for removing impulse noise 
images. The field of research work in optimization is wider as in [25]-[30]. 
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Lena Image 
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Figure 1. Comparing images results of algorithms: (a) denoised images with 70% salt-and-pepper noise, 
(b) recovered images through FR, (c) and (d) restored images using BNN and BTN of 256 * 256 


House image 


Figure 2. Comparing images results of algorithms: (a) denoised images with 70% salt-and-pepper noise, 
(b) recovered images through FR, (c) and (d) restored images using BNN and BTN of 256 * 256 


Elaine image 


Salt-and-pepper noise 
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Figure 3. Comparing images results of algorithms: (a) denoised images with 70% salt-and-pepper noise, 
(b) recovered images through FR, (c) and (d) restored images using BNN and BTN of 256 * 256 


Cameraman picture image 


Salt-and-pepper noise 
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Figure 4. Comparing images results of algorithms: (a) denoised images with 70% salt-and-pepper noise, 
(b) recovered images through FR, (c) and (d) restored images using BNN and BTN of 256*256 
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5. 


CONCLUSION 
Using the second order Taylor series to deriving a new coefficient conjugate for conjugate gradient 


method for image restoration problems. The algorithms exhibit global convergence and the necessary descent 
property holds. In numerical tests, the new method has made significant progress. It demonstrates that the novel 
conjugate gradient approach outperforms the conventional FR conjugate gradient method. 
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